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This  paper  is  the  third  in  a  series  of  reports  concerned  with  the  scaling  of  words  and  phrases  expressing 
qualitative  levels  of  "frequency"  and  "amount."  The  initial  paper  by  Yadrick,  Phalen,  Albert,  Dittmar, 

Weissmuller,  and  Hand  (1993)  replicated  a  study  by  Bass,  Cascio,  and  O'Connor  (1974)  which  used  a  magnitude 
estimation  procedure  to  scale  39  expressions  of  frequency  and  44  expressions  of  amount.  This  study,  however, 
used  Air  Force  basic  trainees  to  provide  the  estimates.  The  resultant  scaling  of  expressions  was  quite  similar  to 
that  of  Bass  et  al.  (1974),  even  though  the  rater  populations  were  quite  different.  This  suggested  that  there  are 
commonly-shared  perceptions  regarding  the  weightiness  of  such  expressions,  independent  of  the  samples  which 
provided  the  estimates.  Nine  expressions  of  amount  were  selected  to  constitute  an  equal  interval  scale,  based  on 
their  magnitude-estimated  weights.  These  were  tested  in  40  computer  administered  occupational  surveys  to  160 
cases  and  were  found  to  produce  more  valid  and  reliable  results  than  the  traditional  nine-point  relative  time  spent 
scale,  as  reported  in  Albert,  Phalen,  Selander,  Dittmar,  Tucker,  Hand,  Weissmuller,  and  Rouse,  (1994). 

The  next  study  by  Yadrick,  Phalen,  Albert,  Dittmar,  Weissmuller,  and  Hand  (1994)  described  the  application  of  a 
new,  univariate  procedure  for  clustering  the  weighted  expressions  into  groups  of  equivalent  (or  synonymous) 
expressions.  It  appeared  to  the  researchers  that  the  expressions  within  each  group  were  sufficiently  synonymous 
with  one  another  that  a  single  expression  might  be  picked  from  each  group  to  represent  an  equal-interval  scale 
with  the  optimal  number  of  points.  It  remained  to  be  determined  whether  the  divisions  or  cut  points  suggested  by 
the  groups  in  the  cluster  solution  had  captured  the  "psychologically"  real  levels  of  a  frequency  or  amount  scale 
embedded  in  the  ordered  lists  of  magnitude-estimated  expressions.  Therefore,  a  validation  study  was  undertaken 
to  determine  the  degree  of  correspondence  between  mathematically  defined  cluster  groups  and  perceptually 
defined,  or  psychologically  real,  groups  of  expressions.  This  paper  will  report  on  the  development  and 
application  of  the  criterion  measure  used  to  validate  the  univariate  clustering  procedure. 


The  Clustering  Procedure 


A  special  measure  of  "equivalence"  was  designed  in  the  previous  study  to  describe  the  similarity  of  expressions 
and  to  cluster  them  into  homogeneous  groups  along  a  single  dimension  ("frequency"  or  "amount").  Since  the 
dimensional  values  for  each  expression  were  derived  by  a  ratio  measurement  technique  (magnitude  estimation), 
equivalence  was  computed  as  a  ratio-based  measure  with  exponential  magnification  of  ratio  differences  to 
accentuate  dividing  lines  between  nonequivalent  sets  of  equivalent  expressions.  In  the  basic  equation,  the 
pairwise  ratios  are  converted  to  logarithms  and  the  logarithms  are  summed  algebraically  so  that  positive  logs 
(ratios  >  1.0)  and  negative  logs  (ratios  <  1.0)  representing  equal  ratio  differences  from  log  1.0  =  0  cancel  each 
other  (compensatory  effect).  Thus,  two  raters  who  disagree  as  to  which  of  two  expressions  is  greater  or  less  will 
negate  each  other’s  estimates.  This  feature  is  contrary  to  standard  similarity  or  overlap  measures,  which  treat  all 
differences  as  representing  dissimilarity  (noncompensatory  effect).  A  detailed  description  of  the  "equivalence" 
equation  with  example  computations  can  be  obtained  from  the  senior  author. 


In  this  study,  the  equivalence  equation  was  applied  first  to  determine  the  equivalence  of  the  top  two  expressions 
in  the  list,  i.e.,  expressions  "1"  and  "2"  on  the  lefthand  side  of  Table  1  for  "frequency"  and  likewise  on  the 
lefthand  side  of  Table  2  for  "amount".  If  the  equivalence  value  exceeded  80.00%,  then  the  equivalence  of 
expressions  "1"  and  "3"  then  "1"  and  "4,"  etc.,  was  computed,  with  expression  "1"  repeatedly  being  used  as  the 
target  expression,  until  the  equivalence  value  fell  below  80.00%.  At  this  point,  the  set  of  expressions  falling 
within  a  minimum  linkage  of  80.00%  was  selected  as  the  first  group  of  expressions  representing  the  highest  level 
of  the  scale.  Thus,  in  Table  1  (lefthand  side)  expressions  "1"  through  "3"  formed  the  first  group  of  equivalent 
frequency  expressions  with  a  minimum  equivalence  (between  expressions  "1"  and  "3")  of  83.81%.  The  next 
target  was  expression  "4".  Expressions  "4"  and  "5"  were  compared,  "4"  and  "6,"  etc.  until  "4"  and  "10"  yielded 
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the  lowest  acceptable  equivalence  of  80.66%.  The  remaining  groups  were  formed  in  a  similar  manner,  resulting 
in  12  "frequency"  groups  (or  scale  levels)  and  13  "amount"  groups  (or  scale  levels).  The  minimum  equivalence 
values  are  reported  to  the  right  of  each  group. 

Column  1  of  Tables  1  and  2  also  shows  the  magnitude-estimated  weights  for  each  expression,  as  derived  by  Bass 
et.  al.  (1974).  Although  the  groups  of  expressions  generated  by  this  clustering  procedure  seemed  to  be  very 
reasonable  in  the  judgment  of  the  authors,  it  could  not  be  assumed  that  the  groupings  represented  psychologically 
real  divisions  without  comparing  them  against  a  criterion  of  psychologically-derived  groups  of  expressions. 

The  Criterion 

A  survey  was  developed  that  contained  the  ordered  lists  of  frequency  and  amount  expressions  (with  their  weights 
deleted)  in  four  different  versions  that  presented  frequency  expressions  first,  followed  by  amount  expressions,  or 
vice  versa,  and  presented  the  expressions  in  high  to  low  order  ("Always"  to  "Never,"  and  "All"  to  "None"),  or 
vice  versa.  The  instructions  asked  respondents  to  follow  a  procedure  that  was  analogous  to  what  was  done  in  the 
clustering  procedure,  but  substituting  his  or  her  psychological  or  perceptual  estimates  of  equivalence  in  place  of 
our  mathematical  calculations  of  equivalence.  More  specifically,  each  respondent  was  asked  to  begin  with 
expression  "  1 "  on  the  list,  which  was  already  circled,  as  the  first  target  expression,  and  to  compare  expression  "2" 
with  it.  If  expression  "2"  appeared  to  be  pretty  much  equivalent  in  meaning  to  expression  "1 ",  the  respondent 
would  proceed  to  compare  expression  "3"  to  expression  "1,"  then  "4"  to  "1"  and  so  forth,  until  reaching  an 
expression  that  did  not  appear  to  be  reasonably  synonymous  or  equivalent  to  expression  "  1 The  respondent 
would  then  circle  this  expression  as  the  next  target  expression  against  which  to  compare  the  expressions 
following  it.  This  procedure  would  continue  until  the  entire  list  was  evaluated.  The  result  would  be  a  set  of 
psychologically-derived  groups  of  expressions  based  on  the  perceptions  of  that  respondent.  In  a  completed 
survey,  each  group  of  expressions  could  be  identified  as  beginning  with  a  circled  expression  and  ending  with  the 
expression  immediately  preceding  the  next  circled  expression. 

A  sample  of  42  respondents,  consisting  of  behavioral  scientists  and  clerical  workers  at  the  Armstrong  Laboratory 
and  at  three  contractor  offices  provided  responses.  Approximately  equal  numbers  of  each  version  of  the  survey 
were  completed  to  provide  the  desired  counterbalancing.  The  groups  identified  by  the  42  respondents  were 
consolidated  into  a  matrix  whose  rows  and  columns  indicated  the  number  of  times  each  expression  was  selected 
as  a  "beginning"  or  "ending"  expression,  respectively.  Evaluation  of  the  row  and  column  totals  made  it  possible 
to  select  the  set  of  groups  which  provided  the  best  overall  fit  of  the  individual  respondent  data.  The  resultant  sets 
.  of  psychologically-defined  groups  based  on  the  perceptual  judgments  of  42  respondents  are  shown  in  the 
right-hand  portion  of  Table  1  for  the  "frequency"  expressions  and  in  the  right-hand  portion  of  Table  2  for  the 
"amount"  expressions.  The  minimum  equivalence  values  for  the  psychologically  defined  groups  are  also  reported 
as  additional  points  of  comparison  with  the  mathematically  defined  groups. 

Frequency  Results 

Table  1  clearly  shows  a  fairly  high  degree  of  correspondence  between  the  mathematical  versus  psychological 
clustering  of  expressions  of  frequency.  Three  groups  are  identical,  and  three  additional  groups  share  one 
boundary  in  common.  Although  the  mathematical  clustering  defined  12  groups  vs  8  groups  for  the  psychological 
clustering,  three  of  the  mathematical  groups  are  single  expressions  which  abutted  sharp  changes  in  the 
magnitude-estimated  weights. 

It  might  be  argued  that  the  expressions  constituting  the  singleton  groups  should  be  dropped,  especially  "seldom," 
which  is  clearly  out  of  place.  This  would  reduce  the  number  of  mathematically  defined  groups  to  nine,  while  the 
psychologically  defined  groups  would  remain  at  eight.  This  would  also  increase  the  correspondence  between  the 
two  clusterings.  A  major  point  of  difference  involves  expressions  "28"  through  "35"  (if  "seldom"  is  eliminated). 
The  mathematical  clustering  separated  this  set  of  expressions  into  three  groups,  while  the  psychological 
clustering  considered  the  set  to  be  one  group.  In  this  case,  the  psychological  grouping  appears  to  make  more 
sense.  However,  it  should  be  noted  that  the  minimum  equivalence  for  this  group  (expressions  "28"  through  "35") 
is  only  0.19,  which  is  to  say  that  there  is  an  extremely  large  ratio  difference  between  a  weight  of  4.72  for  "very 
seldom"  and  a  weight  of  .33  for  "seldom."  If  "seldom"  is  dropped,  the  minimum  equivalence  for  the  group  would 
jump  to  48.84,  which  is  still  low.  Overall,  the  correspondence  between  the  mathematical  and  the  psychological 
clustering  is  reasonably  good,  especially  considering  some  of  the  questionable  expression  weightings. 

Amount  Results 

As  is  evident  from  examining  Table  2,  the  correspondence  between  the  mathematical  and  the  psychological 
clustering  is  not  as  clear  for  expressions  of  amount.  The  greatest  difference  is  at  the  very  top,  where  the  first  four 
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expressions  in  the  psychological  clustering  constitute  four  singleton  groups.  It  appears  that  the  respondents  felt 
that  "all"  was  significantly  more  inclusive  than  "an  exhaustive  amount."  However,  "almost  entirely"  is  certainly 
not  "exhaustive",  and  so  had  to  be  separated  out.  Then  comes  "completely",  which  seems  suspiciously  like  "all" 
(if  adverbs  can  be  like  adjectives),  and  so  "completely"  had  to  be  separated  from  "almost  entirely." 

We  believe  that  the  problem  encountered  here  is  one  of  context.  The  raters  who  made  the  magnitude  estimates 
rated  each  expression  separately,  without  seeing  how  the  expressions  would  ultimately  be  ordered  when  listed 
together,  whereas  the  respondents  in  our  study  had  no  choice  but  to  follow  the  order  in  which  the  expressions 
were  listed  when  they  defined  the  group  boundaries  for  equivalent  expressions.  Again,  the  mathematical 
clustering  defined  more  groups  in  the  mid-  and  low-ranges  than  did  the  psychological  clustering.  The 
psychological  clustering  seems  a  bit  stretched  in  putting  "a  lot"  in  the  same  group  as  "a  moderate  amount",  but  it 
is  harder  to  quibble  with  the  psychological  group  that  includes  expressions  "34"  through  "43". 

Table  1.  Mathematical  and  Psychological  Clustering  Results  for  Expressions  of  Frequency 


Mathematical  Clustering  Psychological  Clustering 


Mag. 

Est. 

Weight 

# 

Frequency  Expression 

Min. 

Eauiv. 

# 

Frequency 

Expression 

Min. 

Eauiv. 

58.01 

1 

Always 

1 

Always 

50.16 

2 

Continually 

2 

Continually 

49.70 

3 

Constantly 

83.81 

3 

Constantly 

83.81 

45.24 

4 

Frequently  if  not 
always 

4 

Frequently  if  not 
always 

42.45 

5 

Very  often 

5 

Very  often 

41.37 

6 

A  great  deal  of  the 
time 

6 

A  great  deal  of  the 
time 

40.02 

7 

Very  frequently 

7 

Very  frequently 

39.28 

8 

A  great  many  times 

8 

A  great  manv  times 

85.35 

39.18 

9 

Usually 

9 

Usually 

37.64 

10 

Often 

80.66 

10 

Often 

36.07 

11 

Frequently 

11 

Frequently. 

35.39 

12 

■  Quite  often 

12 

Quite  often 

34.44 

13 

Rather  frequently 

13 

Rather  frequently 

32.97 

14 

Commonly 

14 

Commonly 

32.64 

15 

Fairly  often 

15 

Fairly  often 

30.65 

16 

Fairlv  manv  times 

83.10 

16 

Fairlv  manv  times 

73.86 

19.42 

17 

Sometimes 

100.00 

17 

Sometimes 

18.01 

18 

Some  of  the  time 

18 

Some  of  the  time 

15.52 

19 

To  some  degree 

19 

To  some  degree 

76.95 

15.19 

20 

Now  and  then 

20 

Now  and  then 
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14.92 

21 

Occasionally 

80.91 

21 

Occasionally 

10.22 

22 

Once  in  a  while 

100.00 

>  22 

Once  in  a  while 

58.91 

7.78 

23 

Not  often 

23 

Not  often 

7.23 

24 

Not  very  often 

24 

Not  very  often 

6.99 

25 

Fairly  infrequently 

25 

Fairly  infrequently 

6.47 

26 

Infrequently 

26 

Infrequently 

6.42 

27 

Rather  seldom 

81.94 

27 

Rather  seldom 

81.94 

4.72 

28 

Very  Seldom 

28 

Very  Seldom 

4.56 

29 

Rarely 

29 

Rarely 

4.54 

30 

Very  infrequently 

96.75 

30 

Very  infrequently 

3.69 

31 

Seldom  if  ever 

31 

Seldom  if  ever 

3.47 

32 

Hardly  at  all 

95.08 

32 

Hardly  at  all 

3.34 

33 

Hardly  ever 

33 

Hardly  ever 

2.99 

34 

Very  rarely 

34 

Very  rarely 

2.63 

35 

Almost  never 

80.77 

35 

Almost  never 

0.33 

36 

Seldom 

100.00 

36 

Seldom 

0.19 

0.17 

37 

None  of  the  time 

37 

None  of  the  time 

0.15 

38 

Not  at  all 

38 

Not  at  all 

\ 

0.08 

39 

Never 

91.69 

39 

Never 

91.69 

Table  2.  Mathematical  and  Psychological  Clustering  Results  for  Expressions  of  Amount 


Mathematical  Clustering 

Psychological  Clustering 

Mag. 

Est. 

Weight 

# 

Amount  Expression 

Min. 

Eauiv. 

# 

Amount  Expression 

Min. 

Eauiv. 

66.12 

1 

All 

1 

All 

100.00 

59.27 

2 

An  exhaustive  amount 

88.70 

2 

An  exhaustive  amount 

100.00 

57.61 

3 

Almost  entirely 

3 

Almost  entirely 

100.00 

57.35 

4 

Completely 

4 

Completely 

100.00 

54.46 

5 

An  extraordinary 
amount 

5 

An  extraordinary 
amount 

51.38 

6 

Almost  completely 

6 

Almost  completely 
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48.89 

7 

An  extremely  abundant 
amount 

7 

An  extremely  abundant 
amount 

48.20 

8 

An  extreme  amount 

81.18 

8 

An  extreme  amount 

87.37 

41.56 

9 

A  great  amount 

9 

A  great  amount 

41.36 

10 

A  great  deal 

10 

A  great  deal 

40.59 

11 

Very  much 

11 

Very  much 

40.50 

12 

A  full  amount 

12 

A  full  amount 

97.45 

37.10 

13 

A  lot 

88.37 

13 

A  lot 

35.14 

14 

Much 

14 

Much 

34.24 

15 

Quite  a  bit 

15 

Quite  a  bit 

32.65 

16 

A  good  bit 

16 

A  good  bit 

31.44 

17 

A  considerable  amount 

17 

A  considerable  amount 

30.04 

18 

Prettv  much 

83.86 

18 

Pretty  much 

27.70 

19 

Fairly  much 

19 

Fairly  much 

26.22 

20 

An  ample  amount 

20 

An  ample  amount 

24.07 

21 

An  adeauate  amount 

85.66 

21 

An  adequate  amount 

21.80 

22 

A  moderate  amount 

22 

A  moderate  amount 

42.40 

18.63 

23 

Some 

84.04 

23 

Some 

13.42 

24 

To  some  extent 

24 

To  some  extent 

13.10 

25 

To  some  degree 

25 

To  some  degree 

11.75 

26 

Somewhat 

87.00 

26 

Somewhat 

51.46 

9.57 

27 

A  limited  amount 

100.00 

27 

A  limited  amount 

7.81 

28 

A  little 

28 

A  little 

7.51 

29 

A  small  amount 

29 

A  small  amount 

7.22 

30 

Comparatively  little 

30 

Comparatively  little 

7.20 

31 

A  little  bit 

31 

A  little  bit 

7.02 

32 

Not  much 

90.19 

32 

Not  much 

5.27 

33 

A  small  degree 

33 

A  small  degree 

41.46 
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34  Very  little 

35  A  slight  amount 

36  A  meager  amount 

37  A  scanty  amount 

38  A  minimum  amount 

39  A  trifling  amount 

40  Scarcely  any 

41  A  trivial  amount 


2.48 


2.28 
0.15 

In  the  mathematical  clustering,  the  singleton  group  containing  "a  limited  amount"  should  perhaps  be  dropped.  It 
seems  to  us  like  another  one  of  those  fuz2y  expressions,  since  all  amounts  other  than  "all"  are  "limited"  amounts. 
Where  the  mathematical  and  psychological  groups  do  not  correspond,  we  would  argue  that  the  mathematical 
grouping  at  the  upper  end  of  the  scaled  list  is  superior,  but  the  psychological  grouping  at  the  lower  end  is 
superior. 

Discussion 

It  appears  to  us  that  the  mathematical  clustering  procedure  used  in  this  study  does  a  reasonably  good  job  of 
clustering  expressions  of  frequency  and  amount  into  groups  of  equivalent  expressions  representing 
psychologically  real  levels  of  frequency  and  amount.  We  plan  to  conduct  a  more  precise  validation  study  when 
time  permits  which  will  allow  us  to  perform  a  statistical  test  of  the  correspondence  between  the  mathematical 
and  the  psychological  clustering  solutions.  This  test  will  consist  of  a  t-test  between  the  mean  correlations  (as 
represented  by  Fisher  Z's)  of  the  42  respondents’  individual  groupings  with  the  mathematically-defined  groups 
and  the  psychologically-defined  groups.  Our  hypothesis  is  that  the  mean  correlations  for  the  mathematical  groups 
will  be  lower,  but  not  significantly  lower,  than  the  mean  correlations  for  the  psychological  groups.  We  might  also 
replicate  the  study  after  removing  from  each  list  those  expressions  that  appear  to  us  to  be  ambiguous  or 
debatably-weighted,  or  we  might  find  or  create  another  set  of  magnitude-scaled  expressions. 
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42 

An  insignificant 
amount 

42 

An  insignificant 
amount 

43 

1 

Hardlv  anv 

82.86 

43 

Hardlv  anv 

29.86 

44 

None 

100.00 

44 

None 

100.00 

5.21 

34 

Very  little 

5.09 

35 

A  slight  amount 

4.28 

36 

A  meager  amount 

81.54 

3.68 

37 

A  scanty  amount 

3.64 

38 

A  minimum  amount 

3.13 

39 

A  trifling  amount 

2.98 

40 

Scarcelv  anv 

82.65 

2.85 

41 

A  trivial  amount 
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